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ALMOST  SURE  CONVERGENCE  OF  ADAPTIVE  IDENTIFICATION 

PREDICTION  AND  CONTROL  ALGORITHMS 
by 

Rajendra  Kumar 

ABSTRACT 

The  paper  is  concerned  with  the  almost  sure  convergence  of  the 
adaptive  parameter  estimation,  N-step  ahead  prediction  and  control 
algorithms  based  upon  standard  least  square  algorithm.  With  the  usual 
stability  and  passivity  assumptions  for  the  prediction  problem,  it  is 
demonstrated  that  the  state  estimation  and  the  N-step  ahead  prediction 
errors  converge  to  the  optimum  such  errors  achievable  with  known  plant 
parameters,  in  the  Cesaro  sense.  An  additional  regularity  assumption  on 
the  signal  model  establishes  the  result  that  the  state  estimation  and 
prediction  errors  also  converge  in  the  strong  sense  at  an  asymptotically 
arithmetic  rate.  Under  an  additional  persistency  of  excitation  condition 
it  is  shown  that  the  parameter  estimation  error  converges  to  zero  at  a 
rate  specified  by  the  degree  of  excitation.  The  persistency  of  excitation 
condition  being  of  a  trivial  nature  is  also  a  necessary  condition  for 
parameter  convergence.  With  the  regularity  condition  holding,  the  con¬ 
vergence  is  also  established  for  the  adaptive  control  algorithms,  e.g. 
self  tuning  regulators  under  the  usual  minimum  phase  restriction  on  the 
plant.  In  this  case  the  tracking  error  equals  the  N-step  ahead  prediction 
error  and  thus  converges  to  its  optimum  value  at  an  asymptotically 


arithmetic  rate. 


1.  INTRODUCTION 


The  paper  is  concerned  with  the  almost  sure  convergence  of  the 
adaptive  parameter  estimation,  N-step  ahead  prediction  and  control  algo¬ 
rithms  based  upon  standard  least  square  algorithm.  With  the  usual  stability 

\ 

and  passivity  assumptions  for  the  prediction  problem,  it  is  demonstrated 
that  the  state  estimation  and  the  N-step  ahead  prediction  errors  converge 
to  the  optimum  such  errors  achievable  with  known  plant  parameters,  in  the 
Cesaro  sense.  Under  an  additional  regularity  assumption,  the  convergence 
of  these  errors  and  also  that  of  the  tracking  error  for  the  adaptive  con¬ 
trol,  is  shown  to  be  in  the  strong  sense  and  at  an  asymptotically  arithmetic 
rate.  An  additional  persistency  of  excitation  condition  of  a  trivial 
nature  also  establishes  the  parameter  convergence. 

In  [1-6]  the  convergence  analysis  for  the  parameter  estimation 
algorithm  focuses  mainly  on  the  consistency  and  asymptotic  properties 
of  the  parameter  estimation  errors.  Of  these  the  most  recent  work  [2] 
establishes  parameter  convergence  and  also  the  convergence  of  the  predic¬ 
tion  error  for  the  stochastic  approximation  algorithm  using  projections 
under  relatively  weak  assumptions  for  such  a  convergence.  Being  concerned 
with  establishing  the  parameter  convergence,  such  analysis  is  bound  to  make 
assumptions  which  are  not  necessarily  satisfied  under  situations  where  the 
parameter  estimation  is  not  the  central  issue,  for  example,  in  the  case  of 
adaptive  prediction  and  control  problems,  see  for  instance  [3,4]. 

Guided  by  this  rationale,  the  work  of  [7-10]  takes  an  alternative 
approach  wherein  the  emphasis  is  placed  on  the  convergence  of  the  predic¬ 
tion  error  in  the  case  of  N-step  ahead  prediction  problem  or  that  of  the 
tracking  error  for  the  case  of  adaptive  control.  In  most  of  these  refer¬ 
ences  the  issue  of  parameter  convergence  is  completely  ignored  except  in 
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[7]  where  simultaneously  some  results  on  the  parameter  convergence  are  also 
established.  An  important  condition  that  emerges  out  from  the  analysis  of 
[4-11]  is  that  a  system  related  to  the  signal  generating  system  or  fre¬ 
quently  just  the  noise  generating  system  be  passive.  In  [11]  unrealistic 
assumption  on  the  adaptive  predictor  termed  "persistency  of  excitation" 
condition  are  also  made  to  derive  convergence.  The  analysis  of  [4]  essen¬ 
tially  ignored  the  global  stability  problem  and  dealt  with  the  local 
behavior  only,  when  dealing  with  the  adaptive  control  problem. 

In  [8]  for  the  adaptive  control  problem,  a  scheme  based  upon  stochastic 
gradient  type  of  parameter  estimation  algorithm  has  been  proposed  along 
with  its  global  convergence  analysis,  thus  generalizing  the  earlier  deter¬ 
ministic  results  of  [14],  In  [7,9,10]  statistically  more  efficient  algo¬ 
rithms  based  upon  modified  least  square  parameter  estimation  schemes  have 
been  proposed  for  the  adaptive  control  and  prediction  problem.  The  modifi¬ 
cation  in  [9]  is  based  upon  the  condition  number  check  of  a  certain  matrix 
so  as  to  ensure  convergence.  In  [7]  the  modification  is  based  upon  a 
stability  measure  and  thus  a  weighted  least  square  algorithm  rather  than 
the  standard  least  square  algorithm  is  used  to  ensure  convergence.  The 
N-step  ahead  prediction  schemes  of  [10]  use  a  bank  of  N  least  square 
parameter  estimation  algorithms  to  ensure  global  convergence.  The 
convergence  in  all  these  references  is  established  in  the  Cesaro  sense. 

The  above  schemes  of  [7-10]  leave  the  question  unanswered  as  to 
whether  or  not  the  schemes  based  upon  the  standard  least  square  algorithm 
converge,  without  any  modification,  whenever  the  modified  schemes  do  so. 

The  question  is  of  more  than  academic  interest.  Whereas  the  schemes  of 
[10]  involve  extra  complexity,  which  is  not  marginal,  the  ones  of  [7]  have 
the  problem  that  these  afford  more  weight  to  the  past  measurements  and 
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thus  the  adaptation  of  these  algorithms  to  time-varying  plants  and  the 
generalization  of  the  convergence  results  may  pose  problems.  In  practical 
implementation,  of  course,  this  problem  can  be  avoidi i  by  several  possible 
schemes  of  ad  hoc  nature.  Furthermore  all  the  schemes  to  date  with  the 
exception  of  [2]  establish  convergence  only  in  the  Cesaro  sense  and  thus 
leave  open  the  possibility  of  the  divergence  to  infinity  of  a  subsequence 
of  the  prediction/tracking  errors  or  even  the  plant  input-output  sequence 
in  case  of  adaptive  control.  The  schemes  also  ignore  the  parameter  con¬ 
vergence  except  in  [7]  where  only  partial  results  on  this  i'ssue  appear. 

The  convergence  analysis  of  this  paper  uses  the  standard  least  square 
algorithm  for  the  parameter  estimation  and  thus  the  results  contain  the 
convergence  of  the  self-tuning  regulators  of  [15,16],  adaptive  predictors 
of  [17]  and  the  adaptive  Kalman  prediction  schemes  of  [11]  as  special  cases. 
The  distinctive  feature  of  the  analysis  of  this  paper  is  that  whereas  in 
the  absence  of  any  regularity  condition, convergence  of  the  prediction  error 
is  established  in  the  Cesaro  sense,  the  regularity  condition  implies  the 
convergence  in  the  strong  sense  and  asymptotically  at  an  arithmetic  rate. 
Furthermore,  this  strong  convergence  of  prediction/tracking  error  is  not  a 
sufficient  condition  for  the  parameter  convergence.  However,  an  additional 
condition  of  a  trivial  nature  also  establishes  the  parameter  convergence 
and  its  rate  of  convergence.  The  condition  is  trivial  because  in  its 
absence  one  cannot  expect  parameter  convergence  with  any  algorithm,  when 
noise  is  present  in  the  measurements.  It  also  emerges  out  from  the  theory 
that  the  convergence  of  parameter  error  need  not  always  be  at  an  arithmetic 
rate.  In  fact  it  could  be  slower  depending  upon  the  excitation,  for 
example,  at  a  logarithmic  rate. 
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In  Section  2,  various  algorithms,  convergence  conditions  and  the 
notations  are  introduced.  The  global  convergence  analysis  is  presented  in 
Section  3  both  for  the  adaptive  N-step  ahead  prediction  and  the  adaptive 
control  problem.  Some  concluding  remarks  are  made  in  Section  4. 


2.  DYNAMICS  OF  MODEL,  ALGORITHM  AND  ESTIMATION  ERRORS 

Although  the  problem  of  one- step  ahead  prediction  forms  a  special  case 
of  the  N-step  ahead  prediction  problem,  for  clarity  of  presentation,  the 
two  are  treated  separately  in  this  section.  This  is  so  because  the  anal¬ 
ysis  of  the  N-step  ahead  prediction/control  algorithm  makes  direct 
application  of  the  results  derived  for  the  one-step  ahead  prediction/control 
problem.  The  development  of  this  section  follows  closely  that  of  [7]. 

Model  Dynamics:  For  the  adaptive  control  of  the  plants  without  a  pure 
delay  or  for  the  one  step  ahead  prediction  problems,  we  consider  the 
following  multi-input  multi-output  model, 

Vl  =  (F  +  Gl0,)xk  +  G2a!k  +  f(uk’Zk)  (2-la) 

zk  =  6'xk  +  “k  (2 . lb) 

where  u^,  z^  and  are  the  inputs,  outputs  and  the  states  of  the  model. 

The  matrices  F,  Gj,  and  the  functions  f  are  assumed  known  with  the 

parameter  matrix  9  and  the  states  x^  unknown.  The  noise  is  a 

zero  mean  white  process  or  more  precisely  is  a  martingale  difference 
sequence  satisfying 

^V^k-l^  =  0  ’  EUI“kll2/Fk.1]  *  °2  <  "  (2.2a) 

l  k 

lim  sup  ~  l  ||w.  ||2  <  »  a.  s .  (2.2b) 

k-*»  i=0 
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where  is  the  minimal  a-algebra  generated  by  As  shown  in 

[7,8],  this  model  includes  multi-input  multi-output  ARMAX  model  with 
exogeneous  inputs  and  also  the  innovation  models. 

When  the  plant  to  be  controlled  has  a  delay  of  less  than  (N+l) 
units  or  when  the  N-step  ahead  prediction  problem  is  considered,  then  a 
model  of  sufficient  generality  is  the  following, 


2 


k 


Xk+N  =  Fxk+N-1  +  Gl6’xk  +  G2nk  +  f(uk’Zk)  (2>3a) 

0’Xk  +  \  *  nk  =  “k+Vk-l  +  ,,'+QN-2Vl-N  (2'3b) 


with  satisfying  (2.2).  The  actual  values  of  the  matrices  are 

not  of  interest  in  the  following  and  F,  are  again  assumed  known 

For  the  ARMAX  models  these  merely  have  as  their  elements,  0  and  1. 

State  and  Parameter  Estimators  The  state  estimator  for  the  model  (2.1) 
is  given,  for  some  estimate  of  the  parameter  matrix  0  at  time  k, 

by 


Vl  =  (F  +  Gl@k)xk  +  G2zk/k  +  f(uk'Zk) 

Zk/k  =  zk  ‘  ®kxk 

and  similarly  for  the  model  (2.3).  The  N-step  ahead  prediction  of 
Zk+N  is  given  by 


(2.4a) 

(2.4b) 


'k+N/k 


0.’  x,  .. 
k  k+N 


(2.4c) 


The  parameter  estimation  algorithm  considered  here  is  the  standard 
least  square  algorithm  for  both  the  one-step  and  N-step  ahead  predictors/ 


controllers. 


6 


0k  "  Vl  *  Fkxkzk/k-l 

(2.5a) 

z,  ,.  ,  =  z.  -  ,  x, 

k/k-1  k  k-1  k 

(2.5b) 

- . 1  „  *  „-i 

P,  .  +  x,  x,'  ,  r,  =  trace  P. 

k-1  k  k  ’  k  k 

(2.5c) 

or 


=  P 


k-1 


^k-lXk(1  +  xkVlV  xkFk-l 


(2 . 5d) 


State  and  Parameter  Estimation  Error  Equations  For  the  one-step  ahead 
predict  ion/control  situation,  the  equation  for  the  state  estimation  error 
is  obtained  from,  (2.1),  (2.4)  and  can  be  reorganized  as 

Xk+1  =  (F +  G0')xk  +  Gqk  ,  qk  =  0^xk  ,  (2.6a) 

pk  ■  ^k  *zsk*k  <2-“» 

The  need  to  reorganize  the  xk  generating  system  in  this  manner,  will  be 
apparent  in  the  sequel.  Similarly  for  the  N-step  ahead  case,  one  obtains 

Vn  =  fVn-i  *  GC?k  +  i  V  (2-7) 


with  pk  and  qk  same  as  in  (2.6).  The  parameter  estimation  error 
9,  =  0  *  obtained  from  (2.5)  is  as  follows. 


0  -  P  X  z 9 

k-1  kkk/k-1 


.  =  8 '  x,  +  0 '  x.  +  uj. 

k/k-1  k  k-1  k  k 


•  ViVk/k 

(2.8a) 

*  K\  *  “k 

(2.8b) 

Passivity  Condition  As  is  common  in  literature  [4-11],  we  make  the 
assumption  that  the  system  (2.6a)  or  (2.7)  with  input  q,  and  output  p 
is  strictly  input  and  strictly  output  passive.  Thus  for  some  k  >  0,  e>0 


* 


and  all  integers  m. 


m 

l  pkqk  ^ 

0  K  K 


m  m 

l  *  -J+  e  l  !lqkl!2  (2.9) 

0  k  k  2  0  k 


We  also  require  that  the  states  of  the  system  (2.6)  or  (2.7)  are 

bounded  in  terms  of  (p^  +  ^  q^) •  For  the  ARMAX  models  this  is  trivially 
satisfied  since  the  matrix  F  has  all  of  its  eigenvalues  at  zero.  For  other 
models  in  addition  to  the  input-output  exponential  stability  implied  by 
(2.9)  we  require  that  the  matrix  F  in  (2.6)  or  (2.7)  has  all  the  eigen¬ 
values  inside  the  unit  circle.  Thus 

r 


m 


+ 


1 

2  qk 


2 


+  < 


some  k  <  °° 


(2.10) 


As  shown  in  [18]  the  passivity  condition  (2.9)  is  equivalent  to  requiring 

that  the  transfer  matrix  {[I-z  ^  ^ 6 ' (z I-F) _1G]  ^  ~  1}  is  strictly 

positive  real.  In  single-input  single -output  ARMAX  model  case  and  for 

N  =  1,  the  condition  is  well  known  and  requires  that  (C  *(z  *)  -  ^  }  be 

strictly  positive  real.  Here  z  denotes  the  Z-  transform  and  C(z_1) 

denotes  in  the  Z-transform  notation,  the  moving  average  polynomial 

associated  with  the  noise  u>.  . 

k 


Stability/Minimum  Phase  Condition:  For  the  open  loop  prediction  problem 
we  require  that  for  some  k  and  all  m>m,  m  being  a  finite  integer, 

£  l  !Ukl!2  $  *  (2.ii) 

m  Q  k 

For  the  closed  loop  adaptive  control  we  require  that  the  trajectory 

* 

z^  is  bounded  and  the  following  minimum  phase  condition  is  satisfied. 
(For  the  N-step  ahead  case  first  sum  in  (2.13)  has  upper  limit  m-N.) 
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1  * 

—  £  ||z,  |j2  s  tc  ,  m  2  m  for  some  <  <  °°  (2.12) 

m  o  * 


m 

*  m  ^  II  zk  ^  2  +  K’  for  some 


tc,  k  <  <*>  (2.13) 


For  the  interpretation  of  (2.13)  in  terms  of  exponential  stability  and  the 
complete  observability/reachability  of  the  inverse  plant  one  may  refer  to 
[7]. 

For  the  convergence  of  the  state  estimation  error  and  the  N-step 
ahead  prediction  error  in  the  Cesaro  sense,  in  the  case  of  N-step  ahead 
prediction  schemes,  the  assumptions  so  far  made  are  sufficient.  However, 
for  the  adaptive  controller  these  assumptions  turn  out  to  be  inadequate  to 
establish  the  convergence  of  the  tracking  error  to  its  minimum  value.  This 
problem  has  been  addressed  in  [7]  where  a  modification  of  the  algorithm 
by  an  appropriate  sequence  of  weighting  coefficients,  ensures  convergence. 
In  [9]  an  alternative  scheme  based  upon  the  condition  number  of  a  certain 
matrix  has  also  been  proposed.  However,  both  these  schemes  prove  the  con¬ 
vergence  in  the  Cesaro  sense  and  thus  leave  open  the  possibility  of  a 

* 

subsequence  of  tracking  errors  diverging  to  Secondly  these  schemes 

do  not  provide  any  analysis  as  to  when  the  standard  least  square  algorithm 
converges.  Furthermore,  whereas  in  [9]  the  problem  of  parameter  conver¬ 
gence  has  not  been  addressed  at  all,  the  condition  in  [7]  required  for 
parameter  convergence  is  of  implicit  nature.  This  is  so  because  the  con¬ 
dition  in  [7]  is  dependent  upon  the  specific  weighting  sequence  selected 
during  a  particular  realization  of  the  process. 


This  is  a  realistic  problem  in  view  of  the  fact  that  one  is  considering 
possibly  unstable  plants  and  in  real  life  situations,  nonlinearities  due 
to  saturation  can  lead  to  overall  divergence.  In  particular,  such  a 
divergence  can  result  if,  even  in  the  absence  of  nonlinearities,  a 
subsequence  of  plant  signals  could  have  high  magnitudes. 


In  this  paper  we  make  an  assumption  termed  the  regularity  condition 


on  the  signal  model  as  follows. 


Regularity  Condition 
k 

l 

k-x°  ‘k  0 


j  K  K 

lim  inf  —  £  x.x!  >  cl  for  some  c  >  0  ,  r,  =  £ 


x.'x. 
1  1 


..  k  +  1 

lxm  sup  — - —  <  °° 

k-*»  rk 


(2. 14a) 

(2.14b) 


Remark  1 .  Condition  (2.14a)  is  a  restriction  on  the  condition  number  of 

* 

the  "covariance"  matrix  associated  with  the  signal  model.  This  condition 
is  expected  to  be  satisfied  if  the  model  has  some  type  of  exponential  sta¬ 
bility  and  is  driven  by  persistent  inputs.  For  the  adaptive  control 

* 

problem,  however,  (2.14)  needs  to  be  interpreted  in  terms  of  and 

(See  the  remarks  at  the  end  of  Section  3  in  relation  to  2.14.) 


3.  CONVERGENCE  ANALYSIS 

The  convergence  results  are  presented  as  three  separate  theorems.  As 
the  analysis  of  the  N-step  ahead  predictor/control ler  makes  direct  use  of 
the  results  for  the  1-step  ahead  predictor/controller,  the  latter  are 
presented  first  in  the  form  of  Theorem  3.1  and  3.2. 

Theorem  3.1:  Consider  the  plant  and  the  least  square  estimation 
scheme  of  the  previous  section  with  noise  condition  (2.2a).  Then,  under 

*  1  r 

The  matrix  lim  -r-  I  x.x!  actually  equals  the  covariance  matrix  of  x, 
k-*»  0  1  1  r' 

provided  that  x^  is  ergodic.  Here,  even  though  x^  is  not  ergodic  we 

are  referring  to  this  matrix  as  "covariance"  matrix. 
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the  passivity  condition  of  Section  2, 
A. 


(i) 


l 


k=0 


\  ^k/k-“kH2  "  “ 


a.s. 


(ii) 


J0  rk‘ikfk-iik  i^k/k'l2 


a.s. 


(iii) 


_-l  It  -  it  2 


l  r,  II  x,  I!  <  » 


k=0 


■k  "Akl 


a.  s . 


Further,  under  the  additional  condition  (2.14)  one  obtains  (3.1a-d)  below, 
wherein  the  condition  (2.14b)  is  required  only  for  the  result  (3. Id)  and  the 

result  (3.1a)  is  also  valid  with  r,  replaced  by  r.  .  For  any  arbitrary  e  >  0, 

K  K. 

B. 

(i)  lim  sup  tr{r,_ G  9,' P,* *9,  }  < 


k-«° 


k  Tk  kJ 


a.  s . 


lim  sup  r£  Uv 
k-x>»  K  K 


“•c)  Pjl2  <  - 


a.s.  (3.1a) 


lim  r,  =  ®  =>  lim  |[9,  ||  =  0 

k-*»  k-*°  k 


a.s. 


(ii) 


V  l'zk/k-wk^  <  * 


l  iy  nx,.ir  < 


k=0 


lk  ,|Ak' 


a.s. 


a.s. 


(3.1b) 


(iii) 


Jo*,|l^-§k-1"2< 


a.s.  (3.1c) 


(iv) 


-e  nr 


rk  11  Vk-1  ‘  “kN‘  ‘ 


a.s.  (3. Id) 
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Proof :  Define  by 

,  k  . . 

Vk  =  tr{0'P’10k}6k  +  {£  [2«iP-qi  -  G<S.(||q.  ||  2  +  ||p.  !|2)]  +  <}  (3.2) 

0 

where  <  and  e  are  the  constants  appearing  in  (2.9),  {6^}  is  some 

1  -  -  -  - 

monotone  nonincreasing  sequence,  pk  =  6'x^  +  j  0£xk  and  qk  =  S^x^. 

Now  5  0  due  to  the  passivity  condition  (2.9)  and  simple  manipulations 

similar  to  those  in  [7]  yield  that 

EtVFk-i!  s  Vi  *  -  eSkEtM2*M2lFk-ii  <3-3> 


where 


\  ■  tr{®kfkl5k-»k-ifk^k-i*2Sk(Pk*“k)'<'k,6k 


?  -n. 


T).  =  6  x'P  X,  |j  2,  ,,  |l2 

k  k  k  k-1  k  "  k/k 

The  latter  equality  follows  from  a  substitution  of  9^  ^  from  the  second 
part  of  (2.8a).  Also  from  the  definition  of  q^  and  the  first  part  of 


(2.8a), 


^Wki'w  ■  25kEh;si;;kiFk-i>  ■  -sk 

ek  ■  25kJkVkE[ll“kl|2|Fk-i' 


(3.4) 


The  substitution  of  the  above  expressions  in  the  second  term  on  the  right- 
hand  side  of  (3.3)  yields 

E[VFk-ii  5  vk-i  -  c5kE'Ki|Z*M2'Fk-ii  *  6k  -  \  <3-5> 


Or  with 


vk  +  E5k(^qk^2  +  +  nk 
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E[viJFk-i]  *  Vi  ‘  c6v_i CMqt 


Jk-iMIMk-i 


*  IIPk-inO  -  Bk  -  Vl  (3.6) 


Now  to  derive  the  results  A  of  the  theorem,  make  a  specific  selec- 


-1 


tion  for  5^,  namely  6^  =  r^  .  Result  (i)  of  Lemma  A2  of  the  Appendix 


then  yields  £  8.  < 00  and  in  view  of  the  positivity  of  V,  and  B,  ,  the 


application  of  the  martingale  convergence  theorem  [21]  implies  that  V 


converges  almost  surely  and  ^  n.  <“,  I  ]]p.  ||  2  r'1  <  °»,  £  ||  q  jj  2  r~ 1  <  °°  a.  s . 


L  HPkir  r'1  -  ,  L  „n.  „ 

0  0  k  okk 

Thus  with  the  application  of  second  part  of  (2.8b)  and  the  definition  of 


yields 


K‘!lvk-“kl'2  -H'llPk'KH*  s  7  1  rk'( ''qk!!2 * 4 II Pj< II <  ”  < 


0 


0 


(3.7) 


l  ^Vk-l^k^k/kH2  "  00 


a.s. 


This  establishes  (i)  and  (ii)  of  result  A.  Now  from  the  bounded  input - 
bounded  state  property  of  the  system,  (2.10),  it  follows  that 


l  Tv1  i  i  xv  1 1 2  <  00 


a.s. 


(3.8) 


To  see  this  note  that  the  system  (2.6a)  can  be  rewritten  as 


-1/2 


-1/2  c  1/2,  -1/2  - 


-1/2  1/2,  -1/2. 


Vi  vi  -  Vi  f 'riv  xk +  c^k:i  rk  v  tPkiv 


has  a  bounded  input-bounded  state  property.  An  intermediate  step  to 
achieve  this  is 


-1/2  ck-i  r-l/2, 

i  ! 


i  ii  vr  f 

o  K  1 


$  I  I|F| 
o 


k-i 


$  K  <  00 


Now  to  derive  the  results  B  of  the  theorem,  it  is  easily  seen  that, 
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or 


and  also 


x,  x,'  <:  2x,  x,'  +  2x,  x,' 

k  k  k  k  k  k 


X,  X  ’  *  X,  X,'  -  X.  X.' 

k  k  2  k  k  k  k 


(3.9a) 


rk  k 


\  *  —  “  l  II !!2  *  r  ^  2r  +  2  ^  |!x.  ||2 

K  *  j  =  0  J  K  k  j  =0  J 

Application  of  (3.8)  and  the  Kronecker  lemma  then  implies  the  existence  of 


an  integer  m^  for  any  given  e^  >  0  such  that 
*"k  1  rk 

—  *^--e  ,  —  <  2  +  e,  for  all  k  £  m. 

rk  2  1  rk  1  1 

Inequalities  (3.9a,b)  imply  that 

k  .  k 


(3.9b) 


—  I  x.x!  *  (4  -  e  )  -^-  t  x.x!  -  —  J  x.x! 

rk  jio  u  2  1  rk  jio  2  ■’  rk  3=0  2  2 


for  k  *  m 


1 


The  condition  (2.14a)  of  Section  2  implies  that  there  exist  an  *  m^ 
such  that  the  first  term  on  the  right-hand  side  of  the  above  inequality 
>  2e2I  for  some  £_,  >  0  and  for  all  k  >,  m7.  An  application  of  (3.8) 
and  the  Kronecker  lemma  implies  that 
k 

l 

lk  j  =0 


T  l  Vj  5  e2r  ’ 

1 v  J  J  * 


k  *  for  some  £  m ^ 


(3.10) 


In  other  words 


or 


-  - 1 

X  .  P, 
mm  k 


X  pv 
max  k 


5  Vk 

for 

k  *  m3 

*  *;V 

for 

k  *  m3 

(3.11) 

(3.6)  with 

6k  5 

r~e  for  some 
k 

e  >  0  and 

now  defining 

.  -  k 

Sk  *  tr{®k^k  Vk^  +  [2rPpiqi  '  GriE;  CllRi  II 2  *  llPi  II 2)  ]  ♦  k,} 

m, 

for  some  0Sx^<»,e>0 
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and 

Sk  -  sk  *  eT'e(||qkii2  *  l!pk!I2)  +  nk 

=  2rk  xk^kxkE^l“k^2lFk-l^  *  ^k  =  rk  xkFk-lxk^  zk/k  H  2 
yields  that  for  k  5 

Etgk|Fk-i]  *  Vi  -  CTke(!l\!l2+  'ipk!!2)  +  ek  -  Yj  (3.i2) 

00 

Application  of  (3.11)  and  the  lemma  A1  yields  that  £  6,  <  00  and 

k=m3 

S.  5  0  as  before.  The  martingale  theorem  is  once  again  applicable  and 

00  -  CO  _ 

thus  Sk  converges  a.s.  and  £  r^Hp^l2  <  00  and  £  rkG^k^2  <  °t>‘  *n 

m3  m3 

a  manner  we  obtained  (3.6),  (3.7),  we  also  get  the  result 


00 


a.s.  (3.13a) 


a.s. 


(3.13b) 

a.s. 


Also,  as  all  the  terms  in  Sk  are  positive,  we  have  that 

lim  sup  <  00  where  denotes  ith  of  the  terms  in  S,  .  Thus 

k-x» 

lim  sup  rke  ®kPj.^®k  <  “  a.s.  (3.14) 

k-w 

which  proves  part  a  of  result  (i) .  With  the  application  of  (3.11),  part  a 
of  result  (i)  implies  (ib).  Since  all  the  signals  are  finite  for  finite  k, 
the  lower  limits  in  (3.13)  can  be  replaced  by  0  thus  yielding  part  (Bii) 
of  the  theorem.  Similarly  the  application  of  (3.11),  (2.8a),  (3.13a)  and 
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r 


I 


definition  of  yields  that 

J  rk-i  H'VVi’112  '* 

k“”3 

As  commented  before,  the  lower  limit  can  once  again  be  replaced  by  0 
thus  implying  (iii) . 

Now  from  (2.4b),  (2.5b),  notice  that 


Zk/k-l  "  Zk/k 


(0,  -  0.  .)  'x, 
k  k-1  * 


Taking  norm  square  on  both  sides  and  applying  part  (i)  of  the  theorem's 
result  B, 


-2e  ||  - 

rk  ^  zk/k-l 


*  2l 


-2e, 


lzk/k-“kl 


2k,  l 


k=m 


x'x, 

44.2 

1+e 


m.  -1 
4 


4  k-1 


Z  11(8 

0 


,  0,  ,)  '  X. 

k  k-r  k1 


for  some  0  <  <  °°  and  some  finite  integer  m^ .  Application  of  (2.14b) 

and  (3.9b)  shows  that  for  m4  sufficiently  large,  rfc  j  in  the  second 
Siam  on  the  right-hand  side  can  be  replaced  by  r^.  Application  of  Lemma  Al, 
(3.1b)  and  the  arbitrary  nature  of  e  establishes  (3. Id).  Further  the 
arbitrary  nature  of  e,  implies  the  strong  convergence  of  various  error 
terms  to  zero.  Also  in  view  of  (3.9b)  r^  can  be  replaced  by  r^  in  (3.1a). 

V  V  V 


Under  the  regularity  condition  (2.14)  of  the  signal  model,  the 
results  B  of  the  theorem  are  end  results  in  themselves.  In  view  of  the 
arbitrary  nature  of  e,  these  imply  that  the  one  step  ahead  prediction 
error  which  is  equal  to  the  tracking  error  for  the  adaptive  con¬ 
troller,  converges  to  at  a  rate  1/k.  Also  for  the  prediction  problem 

the  estimation  error  x^  converges  to  zero  at  a  rate  1/k.  Furthermore 


if  r,  -*■  ®  as  k  approaches  ®,  then  the  parameter  error 


converges 
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to  zero,  the  rate  at  which  this  error  converges  is  given  by  1/r^  or 
with  the  application  of  (3.9b)  by  1/r^.  Moreover,  the  parameter  conver¬ 
gence  does  not  require  (2.14b).  To  obtain  the  result  that  the  inputs  and 
outputs  remain  bounded  for  the  adaptive  control  situation,  we  exploit  the 
minimum  phase  restriction  on  the  plant. 

In  the  absence  of  regularity  condition  (2.14)  the  results  A  of  the 


theorem  cannot  be  extended  to  conclude  the  convergence  of  i  or  t^iat 
the  tracking  error  for  the  case  of  adaptive  control  in  any  sense.  For  the 


prediction  problem,  however,  these  do  denote  some  kind  of  convergence  of 
the  estimation  error  x^.  With  the  stability  assumption  (2.11)  on  the 
signal  model,  as  shown  subsequently,  the  result  (Aiii)  of  the  theorem  also 


implies  the  convergence  of  state  estimation  error  to  zero  and  that  of 
the  prediction  error  Zj,^  \  to  wk  t*le  ^esaro  sense- 

Both  the  sets  of  results  are  given  in  the  form  of  theorem  3.2. 


Theorem  3.2.  Consider  the  adaptive  schemes  of  Section  2  under  the 
passivity  condition  of  the  theorem  3.1,  the  noise  condition  (2.2a,b),  and 
the  stability  assumption  (2.11)  for  the  adaptive  prediction  schemes,  or 
under  the  bounded  trajectory  condition  (2.12)  and  minimum  phase  condition 
(2.13)  for  the  adaptive  control  schemes.  Then  the  result  A  below  holds 
for  adaptive  predictor  in  the  absence  of  (2.14) 


(i) 

(ii) 


lim  sup  ^  X  li x.  ||  2  < 


k-x=  “  0 


Um  l  E  l^i/i  ‘  “ill2  =  0 

k-*»  K  0  1/1  1 


lim  £  l  llx.jl2  =  0 

k-«>  K  0  1 


a.s.  (3.15a) 


a.s. 

(3.15b) 

a.s. 


:srj?rz 


(iii) 
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XkPk-lXk  =  ° 

K-^° 


l  !!(Wi),xk1|4k  *  <  00 


112  -  ~ 1 


a.  s. 


a.  s . 


(3.15c) 


(iv) 


limi  -  o  <3-15d’ 


k-x»  0 


Further,  under  the  regularity  condition  (2.14a)  holding,  then  for  any 
c  >  0,  the  results  (ii)  and  (iv)  in  A  can  be  strengthened  to  (3.15e)  (for 
either  the  adaptive  prediction  or  control  scheme) ,  where  the  last  part  of 
(3.15e)  also  requires  (2.14b). 

B. 


1  k 


0 


'El!W“kl!2  2  * 


I  k‘e  IIO2  < 

o  * 


l  k‘E||5k/k-i-“iJ|2  2 


a.  s . 


a.  s . 


>  (3. 15e) 


a. s.  ) 


Also  for  the  case  of  adaptive  controller  with  (2.14)  holding,  the  states, 
inputs  and  outputs  of  the  plant  remain  bounded. 

C. 


(i) 


lim  sup  r-  l  j(x.  I!' 
k~>  K  0  1 


<  00 


a.  s .  (3. 15f) 


(ii)  lim  sup  r-  l  f|u  !|2  < 


k 

lim  sup 


i  I  !U4il2  -  » 

k-«  K  0  1 


a.  s . 


a.  s . 


(3. 15g) 


Proof:  Considering  the  case  of  adaptive  prediction  first,  the  application 
of  the  Kronecker  lemma  to  result  (Aiii)  of  Theorem  3.1  yields 
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lim  7-  l  !|x. 
k-*°°  k  0 


2  _ 


(3.16) 


Now  the  application  of  ||x^||2  s  2 1]  |j  2  +  2  j|  x.  ||  2  ,  and  (2.11)  results  in 
j  Z  ll-ijll2  -<  y!  IIXjII2.  x2  ,  0  <  «  <  -  (3.17) 


Taking  inverse  on  both  sides  of  (3.17)  and  multiplying  the  resulting  in- 
1  k 

equality  by  £  ||x.  ||2  cm  both  sides 


0 


k 

il  II  x, 
0 


1 2T 1 


r.  F  l\\\\ 

0 


<2  k 


if  z  n^iii2  *  -2r'  ^  i  nx. 


1 

This  inequality  implies  that  lim  sup  r-  1  [|x.  ||2  <  “  a.s.,  for  otherwise 

k-*=°  0  1 

taking  limits  for  a  subsequence,  there  is  a  contradiction  that  Oil,  thus 
establishing  (3.15a).  Substitution  of  (3.15a)  in  the  results  (Ai,iii)  of 
Theorem  3.1  and  the  application  of  the  Kronecker's  lemma  yields  (3.15b). 

First  part  of  (3.15c)  follows  from  the  result  (ii)  of  the  lemma  A2.  Now  premul¬ 
tiplying  second  part  of  (2.8a)  on  both  sides  by  x^  and  taking  norm  square  yields, 


l 


-1 


Now 


+  2 


3k  ’  9k  P'xk"  ’  aPPlicati°n  t*ie 


as  IUk/k.1 1!2  S  2||zk/k| 
result  (Aii)  of  Theorem  3.1  and  the  first  part  of  (3.15c)  then  gives  the 
desired  result  (3.15d)  and  the  second  part  of  (3.15c)  in  view  of  (3.15a). 
For  the  case  of  adaptive  controller  as  shown  in  [7]  for  the  weighted  least 
square  scheme,  one  obtains 


*  T  l  ^i/i-3  — i 


+  K„ 


0  <  <2  <  00 


Further  the  application  of  the  Kronecker's  lemma  to  (3. Id)  yields 
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1  - 

lim  — —  £  ||z.  ,.  .  -a>.||2  =0  which  combined  with  the  above  inequality, 
k-x»  rk  0  1/1-1  1 

once  again  yields  (3.15a).  The  application  of  (3.15a)  in  (3.1b,d)  then, 
results  in  (3.15e).  Applying  [|xi||2<  2 jjxi  |J 2  +  2||xi||2,  (3.15a)  and  the 
second  part  of  (3.15b)  establishes  (3.15f).  The  first  part  of  (3.15g) 
follows  from  (3.15f)  and  the  minimum  phase  condition  (2.13)  whereas  the 
second  part  is  implied  by  (2.1b),  the  noise  condition  (2.2b)  and  (3.15f). 

Result  B  for  the  adaptive  predictor  follows  by  substituting  (3.15a) 
in  (3.1b,d). 

V  V  V 

N-Step  Ahead  Prediction  Schemes  The  convergence  analysis  of  N-step  ahead 
prediction/control  schemes  essentially  involves  the  decomposition  of  the 
problem  into  N  one-step  ahead  predict  ion/ control  problems.  The  direct 
application  of  the  results  of  Theorem  3.1  and  3.2  then  yields  the  desired 
convergence  results. 

Theorem  3.3:  Consider  the  N-step  ahead  prediction/control  schemes  of 
Section  2  under  the  noise  condition  (2.2),  passivity  conditions  (2.9), 
(2.10)  and  the  minimum  phase/bounded  trajectory  conditions  (2.13),  (2.12) 
for  the  closed  loop  adaptive  control  or  the  stability  condition  (2.11) 
for  the  open  loop  prediction  schemes.  Then  in  the  absence  of  condition 
(2.14),  the  results  A  below  hold  for  the  N-step  ahead  predictor, 

A. 

i  k 

(i)  lim  sup  -r-  \  || x.  ]!2  <  ®  a.s.  (3.18a) 

k-~>  K  0  1 

1  k 

(ii)  lim  sup  £  \  [[ 2,/._N  -  n.  || 2  =0  a.s.  (3.18b) 

k-**>  i=0  ' 

Further,  when  the  regularity  condition  (2.14)  is  satisfied,  then  for  both 
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the  prediction  and  adaptive  control  schemes,  one  also  obtains  for  an 
arbitrary  i  >  0, 

B. 

(i)  lim  sup  r^1-£^ll \H 2  <  * 

k-*» 

lim  r,  =  °°  =»  lim  || 0,  jj  =  0 
k-^  K  k-**>  k 


a.s. 

(3.18c) 

a.s. 


(ii) 


l  k‘£  (|x  ||2  <  » 

o  K 

l  k'e  <  " 


a.s. 


(3. 18d) 

a.s. 


(iii)  l  k"C  l|2k/k.N-nkH2  <  00  a-s-  (3. 18e) 

where  part  b  of  (2.14)  is  required  only  for  (3.18e).  Result  (3.18c)  is 
also  valid  with  r^  replaced  by  rfc . 

For  the  N-step  ahead  adaptive  controller,  the  plant  states,  inputs 
and  outputs  are  also  bounded,  i.e.. 


lim  sup  ~  l  ([ x  [{ 2  <  °° 
k-K»  K  0  1 


lim  sup  l  i! u.  )|  2  <  ” 
k-~°  K  0 


lim  sup  r  I  iU;!!2  <  00 
k-«  K  0  1 


a.s. 


a.s. 


a.s . 


}  (3. 18f) 


Proof:  Following  the  procedure  of  [7,18],  the  error  terms  xk*8k  (2. 7, 2. 8) 

are  decomposed  into  N  fictitious  values  as  x£*\  . . .  ,xP*\ 

N  ...  N  ... 

-  r  —  r x  j  **  r  -  (i  J 

with  the  properties  xk  =  l  xk  >  =  2.  ®k 

i=l  i=l 


for  all  k,  where 
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II 

/—\ 

•r-i 

JO 

g(i)  _  p  ~  ~ Ci) ’ 
k-1  kxk  k/k-1 

(3.19a) 

5^  = 

k 

qW  _  p  x  z 
k-1  k-1  kzk/k 

(3.19b) 

;Ci) 

“k/k-l 

=  8’x,(l)  + 

k 

e,(l?x.  +  Q.  Oo).  ,  .  ,Q  .  =  I 

k-i  k  xi-2  k+l-i  X-1 

(3.19c) 

and 


(3. 19d) 


Defining  function  by  replacing  6^,p^,q^  by  »Pj^'  and 

respectively  in  the  obvious  notations,  in  (3.2)  and  in  the  equation  for  V^, 
one  has  the  property  that  E(V^  !  F^]  =  ,  E  1  Fk_i  1  =  °»  as 

Xk’  k’Xke  NC^k-i  ^or  *  =  *>2,...,N.  Working  with  N  supermartingales 
{V^,^  then  an  analysis  similar  to  that  in  Theorem  3.1  yields  that 


l 


-  Q; 


k=0 

l  n1  (|i.(i)il2  < 


k/k  yi-2  k+1 -i  1 


a.  s . 


a.s. 


(3.20a) 


k=0 


L  rk  Xk\- lXk  “k/k  ''  2  < 


k=0 


a.s.  (3.20b) 


From  (3.20a),  by  application  of  the  inequality  j|x.  ||2  s  N  £  11*^  ilz  and 


i  =  l 


the  Kronecker  lemma,  one  obtains 


lira  ~  l  i'x.H2  =  0 
k-x»  rk  0 


a.s.  (3.21) 


The  application  of  stability  condition  as  in  Theorem  3.2  yields  that 
1  k 

lim  sup  r  Il|x.||2  <  <*»  a.s.,  thus  establishing  (3.18a).  This  result,  its 
k-*»  K  0  1 

consequence  that  lim  x'P,  x,  =  0  a.s.  (see  lemma  A2) ,  (3.19b)  and 

k-xx,  K  k 

(3.20b)  then  yield, 
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l  k'1  II  C0^X)  -  §^)1)'xk!|2  < 


a.  s . 


or 


H-r  I  ll«;c§(i)  -8a|)l|2  » 

k-XX>  j=0  ^  3 


a.s. 


Also  from  (3. 19b) , 

N, 


1  V  llt.rst1)  aCi)  -)  j|  2  <  J_  y  !  C 1  p  y  1 

N  k-i  k  k-J  ’  K-j-r1'  '  Ni  k-x ;  kFk-j-l  k-j  !  11  k-i/k-j1 


for  all  i,  j  and  <  <=° 


From  Lemma  A2,  |  x2  .  P.x.  .  [  -*•  0  as  k-*-“  for  all  finite  i  and  i. 

K  + 1  K  K - 1 

Thus  |x£Pk  •  j!  <  e3  f°r  a11  k  %  h’2  anci  ar*y  given  >  0,  N2 
being  some  finite  integer.  Thus  for  >  N, 


+  fl  V  !UCX) 

Ni  n  ♦!  " 


(3.22) 


*»  HJk-j/k-jl|2  -Qi-2“k-j.i-i|!2*2!]l5i-2l!2|lVj.i.ill2  ■ 


Application  of  (3.20a)  and  the  noise  condition  (2.2b)  in  (3.22)  yields,  in 
view  of  arbitrary  nature  of  t. 


1  im 

N  -m> 
1 


SUP  f  l  H^k-j  -^-i-l 

®  1  k  =  l 


=  0  a.s.  (3.23) 


Now  as 


:k/k-N  '  zk/k  +  (6k  ‘  0k-N5 'xk 


(3.24) 


one  obtains 
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1  1  k=l 
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l  -L 

j=0  N1 


N1 

II 

k=l 


i*(8.  .• 

k  k-j 


®k-j-l5 


thus  giving  the  desired  result  (3.18b)  from  (3.18a),  (3.20a)  and  (3.23). 

To  establish  result  B,  proceeding  as  in  the  proof  of  Theorem  3.1, 

under  the  regularity  condition  (2.14a),  r^  in  (3.20)  can  be  replaced  by 
-  £ 

r.  to  obtain 
k 


I  rve  il^kA  -  Qi-2Vl-iH2  " 


k=0 


l  r-MU^II2  < 


k=0 


a.  s . 


a.s. 


(3.25a) 


also  implying  that 


l  C  If  x,  || 2  <  » 

k=0 


CO  _ 

I  rke  H2k/k-"k112 


k=0 


a.s. 


a.s. 


(3.25b) 


Now  the  result  that  lim  sup  tr{e5^  P,  *  9^^}  <  00  a.s.  for  i  =  l, 

k-Ko  k  k  k 

2,  ...,  N  and  the  inequality  (3.11)  following  from  (2.14a)  and  the  first 
part  of  (3.25b)  establish  (3.18c). 

Now  the  condition  (2.14b)  and  (3.9b)  which  results  due  to  the  first 
part  of  (3.25b)  as  in  the  proof  of  theorem  3.1,  imply  that 


-  <  k,  for  all  k  >  N,  ,  0  <  tc_  <  <*>  ,  N.  a  finite  integer  (3.26) 

r,  .  3  3  3  3 

k-1 


Subtracting  n^  from  both  sides  of  (3.24)  and  taking  norm  square  on  both 
sides,  then  application  of  (3.18c)  yields 
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'k/k-N  k 


-  n,  i|  2  S  (N+l)«  z.  ..  -  n,  1 1  2  + 


k/k  "k1 


X  IHVi-Vi-.^JI2} 


and 


j0  ^'»WN-"kll2-<  <»*»(?, rk“»Vk-"k"2*K4  ,J., 


>{i 

^  o 


k=N  .  +  1  t, 
4  k 


N 

N-l  '4 


j  l  Ht®k-i'Sk-i-l),\|l2} 

i=0  k=0  K  1  K  1  1  K  > 


where  is  some  finite  constant  and  N4  =t  is  some  integer.  In  view 
of  (3.25b)  and  the  lemma  A1  the  right-hand  side  of  the  above  inequality  is 
finite,  thus 


l  Tk  l|£k/k-N-\l!2<  “  a.s.  (3-27) 

The  application  of  Kronecker  lemma  and  the  minimum  phase  condition  (2.13) 

rk 

then  implies  lim  sup  -r-  <  «  a.s.,  as  for  the  case  of  adaptive  controller 

k-«°  K 

based  upon  one-step  ahead  prediction,  thus  also  establishing  (3.18a)  for 
the  case  of  the  N-step  ahead  adaptive  control.  Substitution  of  (3.18a) 
in  (3.25b)  and  (5.27)  yields  (3.18d)  and  (3.18e).  Due  to  (3.9b)  which 
also  remains  valid  for  the  N-step  ahead  case,  r^  in  (3.18c)  can  be 
replaced  by  r^.  Results  C  of  the  theorem  are  established  exactly  as  in 
Theorem  3.2,  details  being  omitted  here. 
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Remarks 


1.  As  can  be  inferred  from  the  proof  of  Theorem  3.1,  condition 

(2.14a)  in  the  absence  of  the  stability  condition  (2.11)  is  sufficient  to 
guarantee  that  the  state  estimation  error  x^  and  a  posteriori  prediction 
error  converges  to  zero  in  strong  sense  at  a  rate  1/k. 

2.  For  the  convergence  of  however  in  the  strong  sense  and 

at  a  rate  1/k,  additional  condition  (2.14b)  is  required.  Note  that  a 

sufficient  condition  for  (2.14b)  is  that  lim  sup  [ j x,  1 1  is  finite.  Other 

k-*°° 

sufficient  condition  is  where  lim  inf  =  °°  and  |[x^|(2  or  [|x^j[2 

k-x» 


does  not  increase  faster  than  exponentially  say. 

3.  The  conditions  (2.14)  are  also  sufficient  for  the  convergence 

of  tracking  error  to  zero  at  a  rate  1/k  in  case  of  N-step  ahead  adaptive 
control lers. 

4.  For  the  convergence  of  prediction/tracking  error  to  zero  at  a 
rate  1/k  it  is  not  necessary  that  r^-*-".  In  fact  ^  ma^  very  wel1 
remain  finite  in  the  limit,  thus  the  so-called  persistency  of  excitation 
condition  [11]  is  not  satisfied. 

-  .  A 

5.  In  the  absence  of  persistency  of  excitation,  r^  /  °°,  0^  does 
not  converge  to  0,  however,  the  norm  square  of  the  parameter  estimation 
error  i!  0jJ| 2  is  of  the  order  of  1/r^.  Moreover  if  r^-*-00  then 

9^  0  a.s.  Further,  for  the  convergence  of  parameter,  condition  (2.14b) 

is  not  required. 

6.  For  the  parameter  convergence  it  is  not  important  that  r^ 
approaches  ®  at  a  certain  rate  say  as  k.  In  fact  it  may  do  so  at  a 
very  slow  rate  say,  log  log  k.  In  the  particular  case  when  r^  increases 
at  a  rate  k,  then  ||0^||2  decreases  as  1/k. 


26 


7.  For  the  closed  loop  adaptive  control,  the  regularity  conditions 
need  interpretation.  To  develop  some  intuition  for  these,  consider  the 
case  of  single-input  single-output  ARMAX  model.  Then 

Xk  =  [zk-l  Zk-n’  Vl  •••  Vn’  Vl  •••  Vn]  which  may  be  written  as 

xk  =  [(zk-l  +  zk/k-l3,  •••  (zk-n  + zk-n,k-n-l3’  uk-l  **•  uk-n’  Vl  •”  Vn] ' 
In  view  of  the  minimum  phase  condition,  u^  is  obtained  as  the  output  of  a 

stable  system  with  inputs  and  (z*  +  z^^  ^)  .  If  z^/^  ^  may  be  shown 

to  converge  to  zero  in  some  sense  independent  of  the  condition  (2.14),  then 

(2.14)  will  be  implied  by  a  certain  regularity  condition  on  a  vector 
*  * 

<t>k  =  [z^.i  •••  zk_n>  “fc.i  •••»  ]  ' .  However  the  theory  of  this  paper 

shows  only  the  convergence  of  z^j.  independent  of  (2.14).  This  problem 
is  solved  fully  in  a  subsequent  paper  where  somewhat  different  convergence 
analysis  using  a  priori  prediction  error  method  and  the  truncation  of 
various  estimates  is  used  to  establish  the  complete  results  for  the  adap¬ 
tive  control. 
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4.  CONCLUSIONS 

The  paper  has  presented  the  analysis  demonstrating  under  the  usual 
conditions  of  the  literature,  the  convergence  of  the  adaptive  N-step  ahead 
prediction  schemes  based  upon  the  standard  least  square  algorithm  for 
parameter  estimation,  the  convergence  of  the  prediction  error  being  under¬ 
stood  in  the  Cesaro  sense.  Under  a  regularity  condition  on  the  signal 
model,  it  has  been  further  proved  that  the  convergence  of  the  prediction 
error  and  also  that  of  the  tracking  error  in  the  case  of  adaptive  control, 
is  also  in  the  strong  sense,  with  the  rate  of  convergence  being  asymptoti¬ 
cally  arithmetic.  Under  an  additional  persistency  of  excitation  condition 
of  a  trivial  nature,  the  consistency  of  parameter  estimates  is  also 
established.  The  dependence  of  the  rate  of  parameter  convergence  on  the 
"degree"  of  persistency  of  excitation,  a  scalar  parameter  of  the  signal 
model,  is  also  established.  This  parameter  is  just  the  trace  of  certain 
matrix  in  the  signal  model  and  has  the  intuitive  content  of  signal  power. 

For  the  prediction  problem,  the  regularity  condition  is  a  very 
natural  condition  on  the  signal  model  which  is  already  assumed  to  be  stable. 
For  the  adaptive  control  problem,  this  condition  needs  an  interpretation  in 
terms  of  the  noise  and  the  trajectory  z*,  the  external  inputs  to  the 
closed  loop  controller.  In  a  subsequent  paper  using  a  priori  prediction 
error  method,  the  convergence  analysis  for  the  case  of  adaptive  control  is 
established  with  the  regularity  condition  expressed  directly  in  terms  of 
the  noise  and  the  trajectory  z£. 

The  convergence  results  of  this  paper  have  applicability  to  the  adaptive 
control  of  the  general  non-minimum  phase  plants  of  [22]. 
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APPENDIX 


Lemma  A1 .  For  an  arbitrary  e>0,  with  defined  as  in  (2.5c),  one 

obtains 


V  c.c 

L  k  k  rk 


Proof:  Select  an  integer  m  such  that  e  5  —  and  define  N  =  2m ,  then 
- — -  - —  7m 

the  following  straightforward  manipulations  yield  the  desired  result. 


00  x,  x, 
r  k  k 


V  *  K  <  V  K  x 

1 „  ,1+e  '  A  (l+l/N) 


k=0  r 


k=0  r 


5  (rk-rH-l) 

k.0r<U1/N> 

k 

2mr(l-l/N)  1/N  1/N 

.  v  2  rk  lrk  rk-lJ 

'  k=0  (1+1/N) 

k 


<  7m  y  1  1 

k-o  r1/N  ’  7^ 
K  u  krk-l  rk 


„  _m  -1/N  _ 

£  2  r_^  <  00 

Note  that  to  obtain  the  second  inequality  above,  we  repeatedly  use  the 


identity  (a-b)  =  (*^T  +  /b)  (/a  -  *1>) ;  a,b  i  0. 

Lemma  A2 .  For  the  update  of  given  by  (2.5c,d), 


V  V  V 


00  x.'P,  x, 

r  k  k  k 


y  K  * 

L  y» 

k=0  rk 


Furthermore,  when  lim  sup  -r-  <  00  a.s.,  then  one  obtains 

k-*» 
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(ii)  lim  *£Pkxk  =  0  a .s.,  lim  x^P,  _  xt.  =  0  a.s. 

k-+°°  k-*=°  A 


(iii)  lim  x^+.?.x,  =  0  a.s.  for  all  finite  i.j 

k-x» 


Proof:  Results  (i),  (ii)  follow  from  Reference  [10].  To  obtain  (iii)  use 
the  following  identity  (matrix  inversion  lemma)  repeatedly. 


Pk  = 


k+1 


P,  ,  x,  ,  ( 1 
k+1  k+1 


X»  P  x  ^ 

k+1  k+rk+1 


Y  ’  P 

k+rk+1 


For  example,  for  i  =  2,  i  =  0 


xk+2Pkxk  xk+2Pk+lxk  ^xk+2Pk+lxk+l ^ txk+lPk+lxk} {1  "  xk+lPk+lxk+l } 


-1 


Now 


and 


*Xk+2Pk+lXk'  *  2  ^Xk+2Pk+lXk+2 + XkPk+lXk^ 


^Xk+2Pk+lXk+l '  *2  ^Xk+2Pk+lXk+2  + Xk+lPk+lXk+l^ 


Application  of  these  inequalities  and  result  (ii)  yields  (iii)  for  this 
case.  The  result  for  general  i  and  j  is  similarly  obtained. 
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